From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:14:03 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:02:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:02:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:01:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:47 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 12:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 16 12:01:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:01:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 17 16:01:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:01:03 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:52 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:01:11 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:04:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:04:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:04:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:05:00 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:14 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 16:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 19 16:03:50 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:50 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:51 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:59 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:08:01 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 12:02:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 20 12:02:35 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:35 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:35 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:01:04 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:01:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:01:05 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 21 08:04:18 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:04:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:04:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:04:19 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:15 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:01:03 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:01:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:01:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:01:01 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:03:11 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:01:04 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:42 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:38 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:03:05 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:01:10 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:02:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:02:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:02:20 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:01:05 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 08:03:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 24 08:03:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:38 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 12:03:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 24 12:03:53 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:53 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:53 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:46 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:13 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:31 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:22 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:16 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:17 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:09 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:57 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:24 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 16:01:06 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 26 16:01:07 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:01:07 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:01:07 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:58 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:07:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 08:08:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 27 08:08:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:08:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:08:01 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:08:01 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:31 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 16:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 27 16:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:22 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:13 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:14 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 08:06:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 28 08:06:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:06:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:06:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:06:38 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:09 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:09 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Jan 28 20:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Jan 28 20:01:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:08 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:07:59 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 08:08:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 29 08:08:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:08:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:08:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:08:00 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:19 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:01:08 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:10 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:11:08 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:38 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:49 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:40 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Jan 30 20:02:41 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Jan 30 20:02:41 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:41 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:41 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:41 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:45 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:46 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 12:04:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 31 12:04:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:58 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:58 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:58 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:49 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:27 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:27 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:07 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:11:08 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:11:09 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:11:09 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:03:14 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:03:15 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:23 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:17 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:15:22 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:03:26 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 2 16:01:29 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:29 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:29 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:20 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:14:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:14:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:14:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:14:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 08:14:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 3 08:14:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:14:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:14:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:14:32 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:03:19 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:20 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:25 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:51 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:52 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 08:09:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 4 08:09:53 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:09:53 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:09:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:09:53 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:03:20 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 4 16:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:32 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:23 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:31 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:03:17 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:03:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:03:18 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:26 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:19 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:43 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:17:43 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:43 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:43 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:17:43 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:17:44 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:17:45 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:17:45 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:03:25 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:24 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:21 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:22 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:28:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:28:23 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:04:00 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:04:01 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 16:01:33 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 7 16:01:33 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:33 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:33 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:33 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:26 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:27:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 08:27:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 8 08:27:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:27:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:27:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:27:19 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:04:11 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:04:12 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:04:12 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:35 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:24 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 08:10:35 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 9 08:10:36 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:10:36 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:10:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:10:36 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:55 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:56 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:57 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 16:01:36 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 9 16:01:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:38 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:38 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:38 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:32 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:27:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 08:27:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 10 08:27:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:27:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:27:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:27:24 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:04:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:04:27 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:04:27 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:30 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:32 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:27 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:14:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:14:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:14:29 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:03:19 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:03:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:03:20 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:24 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:15 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:16 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:16 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:10:53 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 12 12:03:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:03:26 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:03:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:03:26 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:28 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:21 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:22 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:20:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:20:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:20:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:20:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:20:26 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:49 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:50 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:48 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:28 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:29 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:17 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:22:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:18 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:22:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:20 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:22 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:22:23 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:22:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:22:26 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:04:05 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:02:02 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:31 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:32 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:34:27 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:29 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 08:34:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 15 08:34:30 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:34:30 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:34:30 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:34:30 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:06:25 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:06:26 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:06:26 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 16:01:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 15 16:01:57 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:57 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:57 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:57 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:34 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:15:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:15:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:53 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:15:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:15:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:54 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Wed Feb 15 20:15:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Wed Feb 15 20:15:56 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:15:56 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:15:56 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:15:56 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:02 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:52:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:03 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:52:04 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:52:06 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Tue Jan 10 20:56:47 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris Message-ID: <20060111025646.GA7520@mail1.thewrittenword.com> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the following on Solaris: $ gdb python core ... #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 1036 rec(mkey) = rec; gdb> bt #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1036 #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, obj=0x1b2da0, recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 #3 0x000220b0 in PyObject_SetItem () #4 0x000b9ba0 in eval_frame () #5 0x000bece8 in PyEval_EvalCodeEx () #6 0x000b6b1c in PyEval_EvalCode () #7 0x000f6d4c in run_node () #8 0x000f6cc8 in run_err_node () #9 0x000f6c68 in PyRun_FileExFlags () #10 0x000f50ac in PyRun_SimpleFileExFlags () #11 0x000f44e8 in PyRun_AnyFileExFlags () #12 0x0001cd40 in Py_Main () #13 0x0001be28 in main () Any ideas? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 15:41:38 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111025646.GA7520@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> Message-ID: <43C51922.7070800@egenix.com> Albert Chin wrote: > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > following on Solaris: > > $ gdb python core > ... > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > 1036 rec(mkey) = rec; > gdb> bt > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1036 > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > obj=0x1b2da0, > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > #3 0x000220b0 in PyObject_SetItem () > #4 0x000b9ba0 in eval_frame () > #5 0x000bece8 in PyEval_EvalCodeEx () > #6 0x000b6b1c in PyEval_EvalCode () > #7 0x000f6d4c in run_node () > #8 0x000f6cc8 in run_err_node () > #9 0x000f6c68 in PyRun_FileExFlags () > #10 0x000f50ac in PyRun_SimpleFileExFlags () > #11 0x000f44e8 in PyRun_AnyFileExFlags () > #12 0x0001cd40 in Py_Main () > #13 0x0001be28 in main () > > Any ideas? Could you try the same with our latest snapshot ?! http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Which compiler did you use to compile the package ? Do you get the same segfaults when compiling the package without optimization ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:22:50 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C51922.7070800@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> Message-ID: <20060111152250.GA38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, > > Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error > > when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the > > following on Solaris: > > > > $ gdb python core > > ... > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > 1036 rec(mkey) = rec; > > gdb> bt > > #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) > > at mx/BeeBase/mxBeeBase/btr.c:1036 > > #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 > > #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, > > obj=0x1b2da0, > > recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 > > #3 0x000220b0 in PyObject_SetItem () > > #4 0x000b9ba0 in eval_frame () > > #5 0x000bece8 in PyEval_EvalCodeEx () > > #6 0x000b6b1c in PyEval_EvalCode () > > #7 0x000f6d4c in run_node () > > #8 0x000f6cc8 in run_err_node () > > #9 0x000f6c68 in PyRun_FileExFlags () > > #10 0x000f50ac in PyRun_SimpleFileExFlags () > > #11 0x000f44e8 in PyRun_AnyFileExFlags () > > #12 0x0001cd40 in Py_Main () > > #13 0x0001be28 in main () > > > > Any ideas? > > Could you try the same with our latest snapshot ?! > > http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Same problem. > Which compiler did you use to compile the package ? GCC 3.4.3. > Do you get the same segfaults when compiling the package > without optimization ? Yes. The above was built with: -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 I started debugging the code and made the following patch to the latest snapshot: --- btr.c.orig Wed Jan 30 13:34:24 2002 +++ btr.c Wed Jan 11 09:20:43 2006 @@ -1034,6 +1034,23 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); + { + char *mkey1; + bRecAddr *mkey2; + bRecAddr val; + + mkey1 = (char *)(mkey) + h->keySize; + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); +printf("mkey1: %lx\n", mkey1); +printf("mkey2: %lx\n", mkey2); + *mkey1 = 0; + *(mkey1 + 1) = 0; + *(mkey1 + 2) = 0; + *(mkey1 + 3) = 1; + val = *mkey2; +printf("*mkey2: %lu\n", *mkey2); +printf("mkey1: %lx\n", mkey1); + } rec(mkey) = rec; childGE(mkey) = 0; ct(buf)++; With this patch the segfault occurs at: #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1050 1050 val = *mkey2; Does this help? -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 09:29:39 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <20060111152939.GB38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 09:22:50AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: > > > Which compiler did you use to compile the package ? > > GCC 3.4.3. Same problem with Python 2.3.5 and the Sun C compiler. -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 17:00:25 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111152250.GA38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> Message-ID: <43C52B99.4010803@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 03:41:38PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> I've built egenix-mx-base 2.0.6 on HP-UX 11.23/PA-RISC, 11.23/IA64, >>> Tru64 UNIX 4.0D, 5.1, IRIX 6.5, and Solaris. All give a SIGBUS error >>> when running the mx/BeeBase/mxBeeBase/test.py script. GDB gives the >>> following on Solaris: >>> >>> $ gdb python core >>> ... >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> 1036 rec(mkey) = rec; >>> gdb> bt >>> #0 0xfee7a104 in bInsertKey (h=0x23f648, key=0x1b2db4, rec=1) >>> at mx/BeeBase/mxBeeBase/btr.c:1036 >>> #1 0xfee73104 in mxBeeIndex_SetKey (self=0x1bbcd0, obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:497 >>> #2 0xfee752c0 in mxBeeIndex_AssignSubscript (self=0x1bbcd0, >>> obj=0x1b2da0, >>> recaddr=0x1cbd90) at mx/BeeBase/mxBeeBase/mxBeeBase.c:1158 >>> #3 0x000220b0 in PyObject_SetItem () >>> #4 0x000b9ba0 in eval_frame () >>> #5 0x000bece8 in PyEval_EvalCodeEx () >>> #6 0x000b6b1c in PyEval_EvalCode () >>> #7 0x000f6d4c in run_node () >>> #8 0x000f6cc8 in run_err_node () >>> #9 0x000f6c68 in PyRun_FileExFlags () >>> #10 0x000f50ac in PyRun_SimpleFileExFlags () >>> #11 0x000f44e8 in PyRun_AnyFileExFlags () >>> #12 0x0001cd40 in Py_Main () >>> #13 0x0001be28 in main () >>> >>> Any ideas? >> Could you try the same with our latest snapshot ?! >> >> http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > > Same problem. > >> Which compiler did you use to compile the package ? > > GCC 3.4.3. > >> Do you get the same segfaults when compiling the package >> without optimization ? > > Yes. The above was built with: > -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -g -mcpu=v8 > > I started debugging the code and made the following patch to the > latest snapshot: > --- btr.c.orig Wed Jan 30 13:34:24 2002 > +++ btr.c Wed Jan 11 09:20:43 2006 > @@ -1034,6 +1034,23 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > + { > + char *mkey1; > + bRecAddr *mkey2; > + bRecAddr val; > + > + mkey1 = (char *)(mkey) + h->keySize; > + mkey2 = (bRecAddr *)((char *)(mkey) + h->keySize); > +printf("mkey1: %lx\n", mkey1); > +printf("mkey2: %lx\n", mkey2); > + *mkey1 = 0; > + *(mkey1 + 1) = 0; > + *(mkey1 + 2) = 0; > + *(mkey1 + 3) = 1; > + val = *mkey2; > +printf("*mkey2: %lu\n", *mkey2); > +printf("mkey1: %lx\n", mkey1); > + } > rec(mkey) = rec; > childGE(mkey) = 0; > ct(buf)++; > > With this patch the segfault occurs at: > #0 0xfee7a4f0 in bInsertKey (h=0x1be978, key=0x188ad4, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1050 > 1050 val = *mkey2; > > Does this help? Are you compiling this on a 64-bit machine ? FWIW: When trying the test.py script on an AMD64 machine it runs through fine. It's been ages since I last looked at the code, so it's hard to tell what the problem could be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 10:20:19 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C52B99.4010803@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> Message-ID: <20060111162019.GC38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > Are you compiling this on a 64-bit machine ? They are all 64-bit CPUs but we are building 32-bit objects. > FWIW: When trying the test.py script on an AMD64 machine it > runs through fine. Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > It's been ages since I last looked at the code, so it's hard > to tell what the problem could be. What I don't understand is why this doesn't work: *mkey1 = 0; *(mkey1 + 1) = 0; *(mkey1 + 2) = 0; *(mkey1 + 3) = 1; val = *mkey2; ^^^^^^^^^^^^^ SIGBUS -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 11:10:14 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111162019.GC38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> Message-ID: <20060111171014.GD38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > > Are you compiling this on a 64-bit machine ? > > They are all 64-bit CPUs but we are building 32-bit objects. > > > FWIW: When trying the test.py script on an AMD64 machine it > > runs through fine. > > Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > > > It's been ages since I last looked at the code, so it's hard > > to tell what the problem could be. > > What I don't understand is why this doesn't work: > *mkey1 = 0; > *(mkey1 + 1) = 0; > *(mkey1 + 2) = 0; > *(mkey1 + 3) = 1; > val = *mkey2; > ^^^^^^^^^^^^^ SIGBUS I think it's a memory-alignment issue. Maybe unsigned longs on the problematic platforms need to be aligned to an even address, or something like that. If so, how to fix? -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 20:27:17 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111171014.GD38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> Message-ID: <43C55C15.3070007@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>> Are you compiling this on a 64-bit machine ? >> They are all 64-bit CPUs but we are building 32-bit objects. >> >>> FWIW: When trying the test.py script on an AMD64 machine it >>> runs through fine. >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >> >>> It's been ages since I last looked at the code, so it's hard >>> to tell what the problem could be. >> What I don't understand is why this doesn't work: >> *mkey1 = 0; >> *(mkey1 + 1) = 0; >> *(mkey1 + 2) = 0; >> *(mkey1 + 3) = 1; >> val = *mkey2; >> ^^^^^^^^^^^^^ SIGBUS > > I think it's a memory-alignment issue. Maybe unsigned longs on the > problematic platforms need to be aligned to an even address, or > something like that. If so, how to fix? The above does hint at that yes. Where exactly in test.py do you get the bus error ? It's possible that on your platforms, a keysize of 10 (which is used in test.py) is not allowed. Perhaps you could try 16 instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 13:43:29 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C55C15.3070007@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> Message-ID: <20060111194328.GE38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: > Albert Chin wrote: > > On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: > >> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: > >>> Are you compiling this on a 64-bit machine ? > >> They are all 64-bit CPUs but we are building 32-bit objects. > >> > >>> FWIW: When trying the test.py script on an AMD64 machine it > >>> runs through fine. > >> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. > >> > >>> It's been ages since I last looked at the code, so it's hard > >>> to tell what the problem could be. > >> What I don't understand is why this doesn't work: > >> *mkey1 = 0; > >> *(mkey1 + 1) = 0; > >> *(mkey1 + 2) = 0; > >> *(mkey1 + 3) = 1; > >> val = *mkey2; > >> ^^^^^^^^^^^^^ SIGBUS > > > > I think it's a memory-alignment issue. Maybe unsigned longs on the > > problematic platforms need to be aligned to an even address, or > > something like that. If so, how to fix? > > The above does hint at that yes. > > Where exactly in test.py do you get the bus error ? #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) at mx/BeeBase/mxBeeBase/btr.c:1037 1037 rec(mkey) = rec; Passing this through the preprocessor: *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; where bRecAddr is an 'unsigned long'. > It's possible that on your platforms, a keysize of 10 (which > is used in test.py) is not allowed. Perhaps you could try 16 > instead. Same error at the same place. I tried 32 but got: Building integer index without duplicates... done. Checking integer index without duplicates... done. Building string index without duplicates... Traceback (most recent call last): File "test.py", line 35, in ? idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) ValueError: illegal sector size (too small or not 0 mod 4) -- albert chin (china@thewrittenword.com) From mal at egenix.com Wed Jan 11 22:35:24 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060111194328.GE38402@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> Message-ID: <43C57A1C.2000208@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 08:27:17PM +0100, M.-A. Lemburg wrote: >> Albert Chin wrote: >>> On Wed, Jan 11, 2006 at 10:20:19AM -0600, Albert Chin wrote: >>>> On Wed, Jan 11, 2006 at 05:00:25PM +0100, M.-A. Lemburg wrote: >>>>> Are you compiling this on a 64-bit machine ? >>>> They are all 64-bit CPUs but we are building 32-bit objects. >>>> >>>>> FWIW: When trying the test.py script on an AMD64 machine it >>>>> runs through fine. >>>> Yeah, we tried on RHEL 3/AMD64 and 4/AMD64 with success. >>>> >>>>> It's been ages since I last looked at the code, so it's hard >>>>> to tell what the problem could be. >>>> What I don't understand is why this doesn't work: >>>> *mkey1 = 0; >>>> *(mkey1 + 1) = 0; >>>> *(mkey1 + 2) = 0; >>>> *(mkey1 + 3) = 1; >>>> val = *mkey2; >>>> ^^^^^^^^^^^^^ SIGBUS >>> I think it's a memory-alignment issue. Maybe unsigned longs on the >>> problematic platforms need to be aligned to an even address, or >>> something like that. If so, how to fix? >> The above does hint at that yes. >> >> Where exactly in test.py do you get the bus error ? > > #0 0xfee7a410 in bInsertKey (h=0x179088, key=0x188c14, rec=1) > at mx/BeeBase/mxBeeBase/btr.c:1037 > 1037 rec(mkey) = rec; > > Passing this through the preprocessor: > *(bRecAddr *)((char *)(mkey) + h->keySize) = rec; > where bRecAddr is an 'unsigned long'. Thanks, but I meant the line in test.py. Still, could you check what the value of mkey is at the time of the bus error ? If this is not aligned properly for the ABI of the platform, it might be causing the problem. The btr.* code was not written by myself and there's a lot of pointer voodoo going on in there, so this will be hard to fix if it's indeed an alignment problem. >> It's possible that on your platforms, a keysize of 10 (which >> is used in test.py) is not allowed. Perhaps you could try 16 >> instead. > > Same error at the same place. I tried 32 but got: > Building integer index without duplicates... done. > Checking integer index without duplicates... done. > Building string index without duplicates... > Traceback (most recent call last): > File "test.py", line 35, in ? > idx = BeeStringIndex(testfile, keysize=32, dupkeys=0, filemode=2) > ValueError: illegal sector size (too small or not 0 mod 4) The message is a bit misleading. There's another case where you get this error: if you choose a keysize that's too large for the sector size. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:09:32 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111220932.GG38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > Thanks, but I meant the line in test.py. for i in xrange(count): try: idx[str(i)] = i+1 except: print ' Problem for key %i: %s' % (i, sys.exc_info()[1]) when i == 0. > Still, could you check what the value of mkey is at the > time of the bus error ? 1BF9AF > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. Yeah, it's way icky. -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Wed Jan 11 16:25:17 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060111222517.GH38402@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > If this is not aligned properly for the ABI of the platform, > it might be causing the problem. It's definitely an alignment problem. On Tru64 UNIX, the error is: ... Unaligned access pid=356725 va=0x1401d44cd pc=0x300008078b8 ra=0x3000080789c inst=0xb5c20000 Unaligned access pid=356725 va=0x1401d44d5 pc=0x300008078c8 ra=0x3000080789c inst=0xb7e10008 ... -- albert chin (china@thewrittenword.com) From egenix-users at mlists.thewrittenword.com Thu Jan 12 01:10:28 2006 From: egenix-users at mlists.thewrittenword.com (Albert Chin) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <43C57A1C.2000208@egenix.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> Message-ID: <20060112071028.GA68561@mail1.thewrittenword.com> On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: > The btr.* code was not written by myself and there's a lot > of pointer voodoo going on in there, so this will be hard > to fix if it's indeed an alignment problem. The attached patch works. Thanks to Gary Vaughan for the idea. It can probably be cleaned up a little more (3rd arg to memcpy()). mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the test suite now passes on all problematic platforms. -- albert chin (china@thewrittenword.com) -------------- next part -------------- Index: mx/BeeBase/mxBeeBase/btr.c =================================================================== --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 @@ -78,10 +78,10 @@ #define eAdr(p) *(bRecAddr *)(p) /* based on k = &[key,rec,childGE] */ -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) #define key(k) (k) -#define rec(k) eAdr((char *)(k) + h->keySize) -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) +#define rec(k) ((char *)(k) + h->keySize) +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) /* based on b = &bBuffer */ #define leaf(b) b->p->leaf @@ -239,6 +239,7 @@ { unsigned int i; bKey *k; + bIdxAddr val; if (!buf) { DPRINTF("\n%s: buf empty\n", msg); @@ -247,13 +248,16 @@ k = key(fkey(buf)); DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", msg, buf->adr, ct(buf), leaf(buf)); - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) DPRINTF(", LT(%04x)", val); if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); DPRINTF("\n"); for (i = 0; i < ct(buf); i++) { + memcpy(&val, rec(k), sizeof(bRecAddr)); DPRINTF(" key %3d: %08x rec(%08x)", - i, *(int *)key(k), rec(k)); - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); + i, *(int *)key(k), val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) DPRINTF(" GE(%04x)", val); DPRINTF("\n"); k += ks(1); } @@ -281,6 +285,7 @@ bError rc; /* return code */ bKey *k; unsigned int i; + bIdxAddr val; if ((rc = readDisk(h, adr, &buf)) != 0) { report(rc); @@ -289,8 +294,10 @@ dumpBuf(h, msg, buf); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - if (childLT(k)) dumpNode(h, msg, childLT(k)); - if (childGE(k)) dumpNode(h, msg, childGE(k)); + memcpy(&val, childLT(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + if (val) dumpNode(h, msg, val); k += ks(1); } return 0; @@ -317,6 +324,7 @@ char p[3*MAX_SECTOR_SIZE]; bBuffer *cbuf, bufx; bBuffer *buf = &bufx; + bIdxAddr val; if (h->sectorSize > MAX_SECTOR_SIZE) { DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); @@ -334,9 +342,10 @@ DPRINTF("\n"); if (ct(buf)) { if (!leaf(buf)) { - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { @@ -346,9 +355,10 @@ _validateTree(h, cbuf, visited, level+1); k = fkey(buf); for (i = 0; i < ct(buf); i++) { - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { - DPRINTF("unable to read buffer %04x\n", childGE(k)); + memcpy(&val, childGE(k), sizeof(bIdxAddr)); + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); + if ((rc = readDisk(h, val, &cbuf)) != 0) { + DPRINTF("unable to read buffer %04x\n", val); return -1; } if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { @@ -410,6 +420,7 @@ int lb; /* lower-bound of binary search */ int ub; /* upper-bound of binary search */ bool foundDup; /* true if found a duplicate key */ + bRecAddr val; /* Test for empty buffer */ if (ct(buf) == 0) { @@ -442,10 +453,11 @@ break; case MODE_MATCH: /* rec's must also match */ - if (rec < rec(*mkey)) { + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); + if (rec < val) { ub = m - 1; cc = CC_LT; - } else if (rec > rec(*mkey)) { + } else if (rec > val) { lb = m + 1; cc = CC_GT; } else { @@ -486,7 +498,7 @@ root = &h->root; gbuf = &h->gbuf; memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); - childLT(fkey(root)) = childLT(fkey(gbuf)); + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); ct(root) = ct(gbuf); leaf(root) = leaf(gbuf); return bErrOk; @@ -513,6 +525,7 @@ int extra; /* extra counts */ int ct; int i; + bIdxAddr z = 0; /* * input: @@ -638,28 +651,28 @@ /* update LT pointer and parent nodes */ if (leaf(gbuf)) { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = 0; + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); /* update parent */ if (i == 0) { - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); pkey += ks(1); } } else { if (i == 0) { /* update LT, tmp[0] */ - childLT(fkey(tmp[i])) = childLT(gkey); + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); /* update LT, parent */ - childLT(pkey) = tmp[i]->adr; + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); } else { /* update LT, tmp[i] */ - childLT(fkey(tmp[i])) = childGE(gkey); + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); /* update parent key */ memcpy(pkey, gkey, ks(1)); - childGE(pkey) = tmp[i]->adr; + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); gkey += ks(1); pkey += ks(1); ct(tmp[i])--; @@ -708,6 +721,7 @@ bError rc; /* return code */ bBuffer *gbuf; bKey *gkey; + bIdxAddr val; /* * input: @@ -728,16 +742,19 @@ /* find 3 adjacent buffers */ if (*pkey == lkey(pbuf)) *pkey -= ks(1); - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; /* gather nodes to gbuf */ gbuf = &h->gbuf; gkey = fkey(gbuf); /* tmp[0] */ - childLT(gkey) = childLT(fkey(tmp[0])); + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); gkey += ks(ct(tmp[0])); ct(gbuf) = ct(tmp[0]); @@ -745,7 +762,7 @@ /* tmp[1] */ if (!leaf(tmp[1])) { memcpy(gkey, *pkey, ks(1)); - childGE(gkey) = childLT(fkey(tmp[1])); + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -756,7 +773,7 @@ /* tmp[2] */ if (!leaf(tmp[2])) { memcpy(gkey, *pkey+ks(1), ks(1)); - childGE(gkey) = childLT(fkey(tmp[2])); + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); ct(gbuf)++; gkey += ks(1); } @@ -949,7 +966,7 @@ if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { if (rec) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); c->buffer = buf; c->key = mkey; return bErrOk; @@ -960,10 +977,16 @@ } } else { if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } } @@ -986,6 +1009,7 @@ bIdxAddr lastGE = 0; /* last childGE traversed */ unsigned int lastGEkey = 0; /* last childGE key traversed */ int height; /* height of tree */ + bRecAddr z = 0; root = &h->root; lastGEvalid = false; @@ -1033,8 +1057,8 @@ /* insert new key */ memcpy(key(mkey), key, h->keySize); - rec(mkey) = rec; - childGE(mkey) = 0; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); ct(buf)++; if ((rc = writeDisk(h, buf)) != 0) return rc; @@ -1045,7 +1069,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), key, h->keySize); - rec(tkey) = rec; + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysIns++; @@ -1059,9 +1083,15 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room in child */ @@ -1073,10 +1103,16 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1122,7 +1158,7 @@ return bErrKeyNotFound; /* update record */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); if ((rc = writeDisk(h, buf)) != 0) return rc; h->nKeysUpd++; @@ -1133,14 +1169,20 @@ /* read child */ if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + bIdxAddr val; + + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } if (cc == CC_EQ) { /* update internal key copy too */ - rec(mkey) = rec; + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); } buf = cbuf; } @@ -1183,7 +1225,7 @@ /* set mkey to point to deletion point */ if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) - *rec = rec(mkey); + memcpy(rec, rec(mkey), sizeof(bRecAddr)); else return bErrKeyNotFound; @@ -1201,7 +1243,7 @@ if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; tkey = fkey(tbuf) + lastGEkey; memcpy(key(tkey), mkey, h->keySize); - rec(tkey) = rec(mkey); + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); if ((rc = writeDisk(h, tbuf)) != 0) return rc; } h->nKeysDel++; @@ -1209,12 +1251,15 @@ } else { /* internal node, descend to child */ bBuffer *cbuf; /* child buf */ - + bIdxAddr val; + /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } /* check for room to delete */ @@ -1237,10 +1282,14 @@ /* read child */ if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } else { - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) + bIdxAddr val; + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; } } @@ -1272,14 +1321,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(fkey(buf)), h->keySize); - if (rec) *rec = rec(fkey(buf)); + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = fkey(buf); return bErrOk; } @@ -1291,14 +1342,16 @@ { bError rc; /* return code */ bBuffer *buf; /* buffer */ + bIdxAddr val; buf = &h->root; while (!leaf(buf)) { - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); + if ((rc = readDisk(h, val, &buf)) != 0) return rc; } if (ct(buf) == 0) return bErrKeyNotFound; if (key) memcpy(key, key(lkey(buf)), h->keySize); - if (rec) *rec = rec(lkey(buf)); + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); c->buffer = buf; c->key = lkey(buf); return bErrOk; } @@ -1328,7 +1381,7 @@ nkey = c->key + ks(1); } if (key) memcpy(key, key(nkey), h->keySize); - if (rec) *rec = rec(nkey); + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); c->buffer = buf; c->key = nkey; return bErrOk; } @@ -1360,7 +1413,7 @@ pkey = c->key - ks(1); } if (key) memcpy(key, key(pkey), h->keySize); - if (rec) *rec = rec(pkey); + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); c->buffer = buf; c->key = pkey; return bErrOk; } @@ -1373,7 +1426,7 @@ if (c->buffer == NULL || !c->buffer->valid) return bErrBufferInvalid; if (key) memcpy(key, key(c->key), h->keySize); - if (rec) *rec = rec(c->key); + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); return bErrOk; } From Ingo_Maurer at gmx.de Fri Jan 13 02:23:32 2006 From: Ingo_Maurer at gmx.de (Ingo Maurer) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge Message-ID: <43C70114.2010302@gmx.de> Hi, how can I install mxODBC on a Debian Sarge system for use with python2.4? First I downloaded the rpm-files (mx-base and mx-commercial) from egenix.com and converted them to deb-files with the debian "alien" software.Then I installed both with debian's dpkg --install. But afterwards when I say in a python-script: import mx.ODBC.iODBC the import statements in the *.py files don't fit. For example often mxODBC ist referenced, but only mx.ODBC (with a dot) works. I uninstalled both mx-packages and I tried to build from source, which I also downloaded from egenix.com, but the following happens: $ python2.4 setup.py install running installerror: invalid Python installation: unable to open /usr/lib/python2.4/config/Makefile (No such file or directory) I have no clue what to do. Thank you for your help Ingo From mal at egenix.com Fri Jan 13 10:29:39 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] mxODBC for python on Debian Sarge In-Reply-To: <43C70114.2010302@gmx.de> References: <43C70114.2010302@gmx.de> Message-ID: <43C77303.70900@egenix.com> Ingo Maurer wrote: > Hi, > > how can I install mxODBC on a Debian Sarge system for use with python2.4? > > First I downloaded the rpm-files (mx-base and mx-commercial) from > egenix.com and > converted them to deb-files with the debian "alien" software.Then I > installed both with > debian's dpkg --install. > > But afterwards when I say in a python-script: > > import mx.ODBC.iODBC > > the import statements in the *.py files don't fit. For example often > mxODBC ist referenced, but only mx.ODBC (with a dot) works. > > I uninstalled both mx-packages and I tried to build from source, which I > also downloaded from egenix.com, but the following happens: > > $ python2.4 setup.py > install > running installerror: invalid Python installation: unable to open > /usr/lib/python2.4/config/Makefile (No such file or directory) > > I have no clue what to do. You need to install the Python -devel package (or whatever development packages are called on Debian). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Jan 13 15:27:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] SIGBUS from mx/BeeBase/mxBeeBase/btr.c on HP-UX, Alpha, IRIX, Solaris In-Reply-To: <20060112071028.GA68561@mail1.thewrittenword.com> References: <20060111025646.GA7520@mail1.thewrittenword.com> <43C51922.7070800@egenix.com> <20060111152250.GA38402@mail1.thewrittenword.com> <43C52B99.4010803@egenix.com> <20060111162019.GC38402@mail1.thewrittenword.com> <20060111171014.GD38402@mail1.thewrittenword.com> <43C55C15.3070007@egenix.com> <20060111194328.GE38402@mail1.thewrittenword.com> <43C57A1C.2000208@egenix.com> <20060112071028.GA68561@mail1.thewrittenword.com> Message-ID: <43C7B8E3.1050805@egenix.com> Albert Chin wrote: > On Wed, Jan 11, 2006 at 10:35:24PM +0100, M.-A. Lemburg wrote: >> The btr.* code was not written by myself and there's a lot >> of pointer voodoo going on in there, so this will be hard >> to fix if it's indeed an alignment problem. > > The attached patch works. Thanks to Gary Vaughan for the idea. It can > probably be cleaned up a little more (3rd arg to memcpy()). > > mx/BeeBase/mxBeeBase/test.py doesn't exercise all of the code but the > test suite now passes on all problematic platforms. Thanks for the patch. We'll see if we can come up with a cleaner way to do this and maybe force the alignment somehow, so that the number of changes necessary is limited. > ------------------------------------------------------------------------ > > Index: mx/BeeBase/mxBeeBase/btr.c > =================================================================== > --- mx/BeeBase/mxBeeBase/btr.c.orig 2001-06-24 06:54:25.000000000 -0500 > +++ mx/BeeBase/mxBeeBase/btr.c 2006-01-12 00:51:46.355167000 -0600 > @@ -78,10 +78,10 @@ > #define eAdr(p) *(bRecAddr *)(p) > > /* based on k = &[key,rec,childGE] */ > -#define childLT(k) bAdr((char *)k - sizeof(bIdxAddr)) > +#define childLT(k) ((char *)k - sizeof(bIdxAddr)) > #define key(k) (k) > -#define rec(k) eAdr((char *)(k) + h->keySize) > -#define childGE(k) bAdr((char *)(k) + h->keySize + sizeof(bRecAddr)) > +#define rec(k) ((char *)(k) + h->keySize) > +#define childGE(k) ((char *)(k) + h->keySize + sizeof(bRecAddr)) > > /* based on b = &bBuffer */ > #define leaf(b) b->p->leaf > @@ -239,6 +239,7 @@ > { > unsigned int i; > bKey *k; > + bIdxAddr val; > > if (!buf) { > DPRINTF("\n%s: buf empty\n", msg); > @@ -247,13 +248,16 @@ > k = key(fkey(buf)); > DPRINTF("\n%s: buf[%04x], ct=%d, leaf=%d", > msg, buf->adr, ct(buf), leaf(buf)); > - if (childLT(k)) DPRINTF(", LT(%04x)", childLT(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(", LT(%04x)", val); > if (leaf(buf)) DPRINTF(", prev(%04x), next(%04x)", prev(buf), next(buf)); > DPRINTF("\n"); > for (i = 0; i < ct(buf); i++) { > + memcpy(&val, rec(k), sizeof(bRecAddr)); > DPRINTF(" key %3d: %08x rec(%08x)", > - i, *(int *)key(k), rec(k)); > - if (childGE(k)) DPRINTF(" GE(%04x)", childGE(k)); > + i, *(int *)key(k), val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) DPRINTF(" GE(%04x)", val); > DPRINTF("\n"); > k += ks(1); > } > @@ -281,6 +285,7 @@ > bError rc; /* return code */ > bKey *k; > unsigned int i; > + bIdxAddr val; > > if ((rc = readDisk(h, adr, &buf)) != 0) { > report(rc); > @@ -289,8 +294,10 @@ > dumpBuf(h, msg, buf); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - if (childLT(k)) dumpNode(h, msg, childLT(k)); > - if (childGE(k)) dumpNode(h, msg, childGE(k)); > + memcpy(&val, childLT(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + if (val) dumpNode(h, msg, val); > k += ks(1); > } > return 0; > @@ -317,6 +324,7 @@ > char p[3*MAX_SECTOR_SIZE]; > bBuffer *cbuf, bufx; > bBuffer *buf = &bufx; > + bIdxAddr val; > > if (h->sectorSize > MAX_SECTOR_SIZE) { > DPRINTF("sectorSize exceeds MAX_SECTOR_SIZE; aborting check\n"); > @@ -334,9 +342,10 @@ > DPRINTF("\n"); > if (ct(buf)) { > if (!leaf(buf)) { > - DPRINTF("level %d: recursing on buf[%04x] LT\n", level, childLT(fkey(buf))); > - if ((rc = readDisk(h, childLT(fkey(buf)), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childLT(fkey(buf))); > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] LT\n", level, val); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(lkey(cbuf)) > *(unsigned int *)key(fkey(buf))) { > @@ -346,9 +355,10 @@ > _validateTree(h, cbuf, visited, level+1); > k = fkey(buf); > for (i = 0; i < ct(buf); i++) { > - DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, key(childGE(k)), i); > - if ((rc = readDisk(h, childGE(k), &cbuf)) != 0) { > - DPRINTF("unable to read buffer %04x\n", childGE(k)); > + memcpy(&val, childGE(k), sizeof(bIdxAddr)); > + DPRINTF("level %d: recursing on buf[%04x] GE[%d]\n", level, val, i); > + if ((rc = readDisk(h, val, &cbuf)) != 0) { > + DPRINTF("unable to read buffer %04x\n", val); > return -1; > } > if (*(unsigned int *)key(fkey(cbuf)) < *(unsigned int *)key(k)) { > @@ -410,6 +420,7 @@ > int lb; /* lower-bound of binary search */ > int ub; /* upper-bound of binary search */ > bool foundDup; /* true if found a duplicate key */ > + bRecAddr val; > > /* Test for empty buffer */ > if (ct(buf) == 0) { > @@ -442,10 +453,11 @@ > break; > case MODE_MATCH: > /* rec's must also match */ > - if (rec < rec(*mkey)) { > + memcpy(&val, rec(*mkey), sizeof(bRecAddr)); > + if (rec < val) { > ub = m - 1; > cc = CC_LT; > - } else if (rec > rec(*mkey)) { > + } else if (rec > val) { > lb = m + 1; > cc = CC_GT; > } else { > @@ -486,7 +498,7 @@ > root = &h->root; > gbuf = &h->gbuf; > memcpy(fkey(root), fkey(gbuf), ks(ct(gbuf))); > - childLT(fkey(root)) = childLT(fkey(gbuf)); > + memcpy(childLT(fkey(root)), childLT(fkey(gbuf)), sizeof(bIdxAddr)); > ct(root) = ct(gbuf); > leaf(root) = leaf(gbuf); > return bErrOk; > @@ -513,6 +525,7 @@ > int extra; /* extra counts */ > int ct; > int i; > + bIdxAddr z = 0; > > /* > * input: > @@ -638,28 +651,28 @@ > /* update LT pointer and parent nodes */ > if (leaf(gbuf)) { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = 0; > + memcpy(childLT(fkey(tmp[i])), &z, sizeof(bIdxAddr)); > > /* update parent */ > if (i == 0) { > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof (bIdxAddr)); > pkey += ks(1); > } > } else { > if (i == 0) { > /* update LT, tmp[0] */ > - childLT(fkey(tmp[i])) = childLT(gkey); > + memcpy(childLT(fkey(tmp[i])), childLT(gkey), sizeof(bIdxAddr)); > /* update LT, parent */ > - childLT(pkey) = tmp[i]->adr; > + memcpy(childLT(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > } else { > /* update LT, tmp[i] */ > - childLT(fkey(tmp[i])) = childGE(gkey); > + memcpy(childLT(fkey(tmp[i])), childGE(gkey), sizeof(bIdxAddr)); > /* update parent key */ > memcpy(pkey, gkey, ks(1)); > - childGE(pkey) = tmp[i]->adr; > + memcpy(childGE(pkey), &tmp[i]->adr, sizeof(bIdxAddr)); > gkey += ks(1); > pkey += ks(1); > ct(tmp[i])--; > @@ -708,6 +721,7 @@ > bError rc; /* return code */ > bBuffer *gbuf; > bKey *gkey; > + bIdxAddr val; > > /* > * input: > @@ -728,16 +742,19 @@ > /* find 3 adjacent buffers */ > if (*pkey == lkey(pbuf)) > *pkey -= ks(1); > - if ((rc = readDisk(h, childLT(*pkey), &tmp[0])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey), &tmp[1])) != 0) return rc; > - if ((rc = readDisk(h, childGE(*pkey + ks(1)), &tmp[2])) != 0) return rc; > + memcpy(&val, childLT(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[0])) != 0) return rc; > + memcpy(&val, childGE(*pkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[1])) != 0) return rc; > + memcpy(&val, childGE(*pkey + ks(1)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &tmp[2])) != 0) return rc; > > /* gather nodes to gbuf */ > gbuf = &h->gbuf; > gkey = fkey(gbuf); > > /* tmp[0] */ > - childLT(gkey) = childLT(fkey(tmp[0])); > + memcpy(childLT(gkey), childLT(fkey(tmp[0])), sizeof(bIdxAddr)); > memcpy(gkey, fkey(tmp[0]), ks(ct(tmp[0]))); > gkey += ks(ct(tmp[0])); > ct(gbuf) = ct(tmp[0]); > @@ -745,7 +762,7 @@ > /* tmp[1] */ > if (!leaf(tmp[1])) { > memcpy(gkey, *pkey, ks(1)); > - childGE(gkey) = childLT(fkey(tmp[1])); > + memcpy(childGE(gkey), childLT(fkey(tmp[1])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -756,7 +773,7 @@ > /* tmp[2] */ > if (!leaf(tmp[2])) { > memcpy(gkey, *pkey+ks(1), ks(1)); > - childGE(gkey) = childLT(fkey(tmp[2])); > + memcpy(childGE(gkey), childLT(fkey(tmp[2])), sizeof(bIdxAddr)); > ct(gbuf)++; > gkey += ks(1); > } > @@ -949,7 +966,7 @@ > > if ((cc=search(h, buf, key, 0, &mkey, MODE_FIRST)) == CC_EQ) { > if (rec) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > c->buffer = buf; > c->key = mkey; > return bErrOk; > @@ -960,10 +977,16 @@ > } > } else { > if (search(h, buf, key, 0, &mkey, MODE_FIRST) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &buf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) > return rc; > } > } > @@ -986,6 +1009,7 @@ > bIdxAddr lastGE = 0; /* last childGE traversed */ > unsigned int lastGEkey = 0; /* last childGE key traversed */ > int height; /* height of tree */ > + bRecAddr z = 0; > > root = &h->root; > lastGEvalid = false; > @@ -1033,8 +1057,8 @@ > > /* insert new key */ > memcpy(key(mkey), key, h->keySize); > - rec(mkey) = rec; > - childGE(mkey) = 0; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > + memcpy(childGE(mkey), &z, sizeof(bRecAddr)); > ct(buf)++; > if ((rc = writeDisk(h, buf)) != 0) return rc; > > @@ -1045,7 +1069,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), key, h->keySize); > - rec(tkey) = rec; > + memcpy(rec(tkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysIns++; > @@ -1059,9 +1083,15 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room in child */ > @@ -1073,10 +1103,16 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1122,7 +1158,7 @@ > return bErrKeyNotFound; > > /* update record */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > if ((rc = writeDisk(h, buf)) != 0) return rc; > > h->nKeysUpd++; > @@ -1133,14 +1169,20 @@ > > /* read child */ > if ((cc = search(h, buf, key, rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + bIdxAddr val; > + > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > if (cc == CC_EQ) { > /* update internal key copy too */ > - rec(mkey) = rec; > + memcpy(rec(mkey), &rec, sizeof(bRecAddr)); > } > buf = cbuf; > } > @@ -1183,7 +1225,7 @@ > > /* set mkey to point to deletion point */ > if (search(h, buf, key, *rec, &mkey, MODE_MATCH) == CC_EQ) > - *rec = rec(mkey); > + memcpy(rec, rec(mkey), sizeof(bRecAddr)); > else > return bErrKeyNotFound; > > @@ -1201,7 +1243,7 @@ > if ((rc = readDisk(h, lastGE, &tbuf)) != 0) return rc; > tkey = fkey(tbuf) + lastGEkey; > memcpy(key(tkey), mkey, h->keySize); > - rec(tkey) = rec(mkey); > + memcpy(rec(tkey), rec(mkey), sizeof(bRecAddr)); > if ((rc = writeDisk(h, tbuf)) != 0) return rc; > } > h->nKeysDel++; > @@ -1209,12 +1251,15 @@ > } else { > /* internal node, descend to child */ > bBuffer *cbuf; /* child buf */ > - > + bIdxAddr val; > + > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) return rc; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) return rc; > } > > /* check for room to delete */ > @@ -1237,10 +1282,14 @@ > > /* read child */ > if ((cc = search(h, buf, key, *rec, &mkey, MODE_MATCH)) == CC_LT) { > - if ((rc = readDisk(h, childLT(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childLT(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } else { > - if ((rc = readDisk(h, childGE(mkey), &cbuf)) != 0) > + bIdxAddr val; > + memcpy(&val, childGE(mkey), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &cbuf)) != 0) > return rc; > } > } > @@ -1272,14 +1321,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childLT(fkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childLT(fkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(fkey(buf)), h->keySize); > - if (rec) *rec = rec(fkey(buf)); > + if (rec) memcpy(rec, rec(fkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = fkey(buf); > return bErrOk; > } > @@ -1291,14 +1342,16 @@ > { > bError rc; /* return code */ > bBuffer *buf; /* buffer */ > + bIdxAddr val; > > buf = &h->root; > while (!leaf(buf)) { > - if ((rc = readDisk(h, childGE(lkey(buf)), &buf)) != 0) return rc; > + memcpy(&val, childGE(lkey(buf)), sizeof(bIdxAddr)); > + if ((rc = readDisk(h, val, &buf)) != 0) return rc; > } > if (ct(buf) == 0) return bErrKeyNotFound; > if (key) memcpy(key, key(lkey(buf)), h->keySize); > - if (rec) *rec = rec(lkey(buf)); > + if (rec) memcpy(rec, rec(lkey(buf)), sizeof(bRecAddr)); > c->buffer = buf; c->key = lkey(buf); > return bErrOk; > } > @@ -1328,7 +1381,7 @@ > nkey = c->key + ks(1); > } > if (key) memcpy(key, key(nkey), h->keySize); > - if (rec) *rec = rec(nkey); > + if (rec) memcpy(rec, rec(nkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = nkey; > return bErrOk; > } > @@ -1360,7 +1413,7 @@ > pkey = c->key - ks(1); > } > if (key) memcpy(key, key(pkey), h->keySize); > - if (rec) *rec = rec(pkey); > + if (rec) memcpy(rec, rec(pkey), sizeof(bRecAddr)); > c->buffer = buf; c->key = pkey; > return bErrOk; > } > @@ -1373,7 +1426,7 @@ > if (c->buffer == NULL || !c->buffer->valid) > return bErrBufferInvalid; > if (key) memcpy(key, key(c->key), h->keySize); > - if (rec) *rec = rec(c->key); > + if (rec) memcpy(rec, rec(c->key), sizeof(bRecAddr)); > return bErrOk; > } > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 13 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Jan 18 18:02:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:34:37 2006 Subject: [egenix-users] Re: mxTextTools issues In-Reply-To: <43CE669A.9060703@wilcoxon.org> References: <43CE669A.9060703@wilcoxon.org> Message-ID: <43CE749C.9010905@egenix.com> Scot Wilcoxon wrote: > I tried to use mxTextTools in a pywikipedia bot (Python script for > improving Wikipedia.org articles). I thought I'd point out several > issues I noticed. > > 1. No co-nested patterns. I needed to handle several patterns which > could be nested within other patterns, for example [[...]] could contain > {{...}} and vice versa. Having parsing rules for those two patterns > worked OK, but I could not seem to create a forward declaration so the > parsing table which was declared first could refer to the second parsing > table. (Creating a null or simple table for just seemed to include that > in the first table rather than being replaced with the second table when > it was defined.) You can have nesting by using nested tables. These then work recursively, do rollback, etc. > 2. Painful debugging. I realize when mxTextTools was created that > Python did not have the Logging module available. It is hard to find > how parsing is proceeding and figuring out why a parse pattern is not > being recognized. It doesn't help that the tree-printing function > isn't documented nor how to invoke it (I had to figure it out from the > source code). > > 3. Infinite loops. Yes, I found the Charming Python jump_count loop > detector. mxTextTools uses a very low-level state machine to do the actual parsing - that's why it's so fast, but also makes programming the patterns a bit cumbersome. That's the price to pay for performance, I guess. If you want low-level tracing, you'll have to compile mxTextTools with debugging enabled. If you then run Python with -d command line flag, the module will create a log file mxTextTools.log which has lots and lots of details. Compiling the debug version is easy: python setup.py mx_autoconf --enable-debugging install The 2.1.0 version of mxTextTools makes writing tag tables a lot easier by supporting jump target strings and is fully Unicode aware. This is our latest snapshot: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip > 4. The documentation for WordStart and WordEnd at one point (I don't > remember where) does not make the difference between them apparent. WordStart leaves the head on the first char of the word, WordEnd on the last. > 5. Community. This email address is the only apparent contact point. A > public forum of some sort might help make it apparent whether > mxTextTools is a live or dead project, would allow people to help each > other, and people with problems could search for similar previously > solved problems. egenix-users is our user mailing list, you can use that as forum. We usually listen to what our users have to say :-) The archives are also scanned by Google, so it's easy to do searches. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::